Facilitation of search, list, and retrieval operations on persistent data set using distributed shared memory

• US 8,819,059 B2
• Filed: 06/04/2012
• Issued: 08/26/2014
• Est. Priority Date: 11/19/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aronovich, Lior, Levy, Asaf, Loya, Liran
• Primary Examiner(s): Chau, Dung K
• Application Number:
  US13/488,150
• Publication Number:
  US 20120239687A1
• Time in Patent Office: 813 Days
• Field of Search: None
• US Class Current: 707/769
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/2023 Failover techniques
• CPC Class Codes: G06F 11/2043 where the redundant components share a common memory address space
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/825 the problem or solution involving locking
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2209/523 Mode
• CPC Class Codes: G06F 3/121 Facilitating exception or error detection and recovery, e.g. fault, media or consumables depleted
• CPC Class Codes: G06F 9/3863 using multiple copies of the architectural state, e.g. shadow registers
• CPC Class Codes: G06F 9/526 Mutual exclusion algorithms
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

In one exemplary embodiment, a DSM is used to store the data structures in a cluster shared memory, such that updates, deletions and insertions are applied to both the persistent data set and the DSM, and search, list and retrieval operations are processed directly from the DSM. A data structure in the persistent data set is stored in two separate DSM pages: a first page includes an identifying search key of the data structure, the search key also pointing to a location of the data structure in the storage. A second page includes the updatable properties of the data structure. A DSM storing the data structures'"'"' pages is organized as a two-row matrix.

Claims

1. A method for facilitating search, list and retrieval operations on a persistent data set using a distributed shared memory (DSM) by a processor device, comprising:

using the DSM to store a plurality of data structures, such that updates, deletions and insertions are applied to both the persistent data set and the DSM, and the search, list and retrieval operations are processed directly from the DSM;

storing a data structure of the plurality of data structures in at least two separate pages in the persistent data set by;

defining a first DSM page to include an identifying search key of the data structure, and

defining a second DSM page to include at least one updatable property of at least one of the plurality of data structures;

wherein the first and second DSM pages are assigned as an occupied column in a two-row matrix in the DSM, and the identifying search key is stored in a first row of the two-row matrix and the at least one updatable property is stored in the second row of the two-row matrix such that in each occupied column of the two-row matrix corresponds to a specific data structure in the persistent data set;

pursuant to a search operation of the data structure of the plurality of data structures, performing each one of;

taking a shared permission on a synchronization page for a duration of the search operation,

performing the search using at least one shared permission to at least one first DSM page via the identifying search key, and

releasing the shared permission to the synchronization page via the identifying search key, and subsequently, to the synchronization page, following a completion of the search operation; and

pursuant to a modification operation of the data structure of the plurality of data structures, performing each one of;

taking the shared permission on the synchronization page for a duration of the modification operation,

performing the search using the shared permission to at least one first DSM page via the identifying search key,

taking exclusive permission on the synchronization page containing updatable portions of the data structure,

updating the synchronization page and updating the data structure associated with the synchronization page, and

releasing the exclusive permission to the synchronization page via the identifying search key, and releasing the shared permission on the synchronization page, following a completion of the search operation.

2. The method of claim 1, wherein the identifying search key points to a location of the data structure in the persistent data set.

3. The method of claim 2, further including defining the synchronization page in the DSM, associated with the DSM matrix, to facilitate synchronization between operations modifying an assignment of at least one column in the DSM matrix to a persistent data structure, and operations that do not modify such assignment.

4. The method of claim 3, wherein for adding a data structure an exclusive permission is taken on the synchronization page for a duration of the operation, and for all other types of operations a shared permission is taken on the synchronization page for the duration of the operation.

5. The method of claim 3, further including, pursuant to a subsequent search operation for an additional data structure of the plurality of data structures, performing at least one of:

performing a search using at least one shared permission on at least one first DSM page, and

following a completion of the search operation, releasing the shared permission to the at least one DSM page, and subsequently, to the synchronization page.

6. The method of claim 3, further including, pursuant to a subsequent modification or deletion operation for an additional data structure of the plurality of data structures, performing at least one of:

performing a search on at least one first DSM page to locate the additional data structure,

performing one of taking an exclusive permission on the second DSM page if modifying the additional data structure and taking the exclusive permission on the first DSM page if deleting the additional data structure,

performing one of modifying the second DSM page if modifying the additional data structure and marking the first DSM page as not assigned if deleting the additional data structure,

performing one of modifying an associated persistent data structure if modifying the additional data structure and removing the associated persistent data structure from a location in persistent storage if deleting the data structure, and

releasing the exclusive permission on one of the first and the second DSM page, and subsequently, releasing the shared permission on the synchronization page.

7. The method of claim 3, further including, pursuant to a subsequent insertion operation for a new data structure, performing at least one of:

taking an exclusive permission on the synchronization page for a duration of the insertion operation,

performing a search on at least one first DSM page to locate an appropriate location for the new data structure to be inserted,

moving data structures of the plurality of data structures to create an empty column for the new data structure,

inserting the new data structure, by taking exclusive permissions on the first and second pages of the empty column assigned to the new data structure,

updating an entry in a DSM recovery file with at least one key property of the new data structure,

writing the new data structure to persistent storage, and

releasing the exclusive permission on the first and second DSM pages, and subsequently, releasing the exclusive permission on the synchronization page.

8. The method of claim 1, further including defining a DSM recovery file in a persistent storage, the DSM recovery file organized such that an entry in the file corresponds by a location to a column in the DSM matrix.

9. The method of claim 8, wherein the DSM recovery file is updated during insertion operations, and is used during failover processing to locate persistent data structures in the persistent storage to be loaded into invalid DSM pages.